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ABSTRACT 

A Study was carried out to ascertain whether language 
instructors or naive judges were more reliable in judging oral 
proficiency* Fifteen students were chosen from the ESL (English as a 
second language) center at Southern Illinois Oni versify to record a 
tape while reading passages in English. The tape and a questionnaire 
were administered to 70 raters* Half of them' were students who had 
neither linguistic nox teaching experience (the naive group) while 
the other half were instructors or teaching assistants in ESL (the 
experienced group)* Each sample was rated by the judges on each of 
six scales: th4 first four scales consisted of four pairs of bipolar 
adjefctival descriptors, an overall prof iciency ^scale, and a multiple 
choice questionnaire about the language background of the speaker* 
There was a very substantial agreement among the raters, regardless 



of wl ether they 
reliable on the 



were naive or experienced* Both groups were very 
whore in judging proficiency level, although the 

The experienced raters were 
source language backgrounds 



experienced were sos^ewhat more reliable* 
much better at identifying correctly the 
than the naive group* (CFH) 
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Accent and the Evaluation of ESL Oral Proficiency 



Donn R. Callaway 
Southern Illinois University 

fhe evaluation of oral proficiency in a second language has 
always been a formidable problem for the second langua^^e teacher. 
The teacher mast accurately evaluate the leeirner'o ability ^'to 
receive or transmit information in the test language for saae 
pragmatically useful purpose" (Clark, 1975, 10).^ However, 
in order to evaluate the student's oral proficiency the teacher 
must r^ly on higl:ily subjective judgements of the student's output. 

Studies of evaluative reactions to speech samples by Lam- 
bert (i960), Labov (1966), and others have supported the notion 
that a linguistically-naive listener can and does make critical 
evaluations .of the speaker's personality, social class, or 
ethnicity* These studies h&ve used either variations between 
dialects or between languages as the independent variable. 
While the dependent variables have consisted of a general per- 
sonality factor, a social class ratings individual personality 
traits, or separate speech characteristics. Since, in these 
studies, speech characteristics tended to be ignored, V/illioms 
called for research: 

to link whatever language and speech 
features serve as salient cues in 
this judgmental process with whatever 
kinds of evaluation or stereotypes are 
of interest to us in the behavior of 
listeners (1970, ^73). 
Williams showed that social class and Judgement ratings can be 
predicted from the presence, absence, or strength Cf certain 
language features, among them silent pausing and verb constructions 
(1970, ^77). However, since the study was limited to the effect 
of Black dialect on Black and White elementary school teachers 
and did not include other dialects and other evaluators, further 
research is needed. 

As a result of the concern of sociolinguists that accented 
speech can cause alienation and discriminatioci in educational 
and occupational opportunities (Ortego, 1970: Ryan, 1973), 
most studies have dealt only with the language varieties of 
various minority groups: French Canadians (Lambert, Hodgson, 
Gardner, and Fillenbaum, I96O; Anisfeld and Lambert, I96U; 
Webster and Kramer, I968), Black Americans (Harms, 1963; 
Shuy, Baratz, and Wolfram, 1969; Tucker and Lambert, 1969; 
Williams, 1970; Williajiis, Whitehead, and Miller, 1971), Mexican 
Araericans (Ortego, 1970; Willijms, Whitehead, and Miller, 1971^ 
Ryan, 1973), ari British regionals (Strongman and Woosley, 1967; 
Giles, 1972). Ricnards points out that any: 

Deviancy from grammatical or phonological 
norms of a speech community elicits 
evaluational reactions that may classify 
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a person unfavorably (1971, 21). 
If this divergence frcm the standard language has an effect 
on these minorities, then it could have an even more pronounced 
effect on the acceptance of the second language learner. 

In oral evaluation, a general assumption is that any native 
speaker can assess the proficiency of a non-native. In fact, 
a generally used measure, the American Language Institute Oral 
Rating From, describes a speaker of minimum proficiency as 
having "pronunciation... virtually unintelligible to 'the mm 
in the street Lw italicsD' "(1962). However, usually, it is 
the second language teacher, rather than 'the man in the Street 
who makes the evaluation. Is it safe to assume that there is 
no significant difference between a trained rater and an untrained 
one^ Cartier does not think it is. He says that Judgements 
of proficiency 

are made by the wrong people, they are 
made by sophisticated language instructors 
who have become q.uite skilled at under- 
standing heavily dialectal Engl J rather 
than the student's eventual instructors, 
classmates, and Job supervisors (1958, 21). 

He aeems to be implying that naive Judges might be better. 

According to Jakobovits (1970), however, naive Judges are apt. 

to attribute too much importance to "accent, pronunciation, and 

fluency" and too little to the weightier matters of grammar and 

vocabulary (85). , , ^-p 

' A number of research studies have .examined the effect or 
accent in bidialectal and bilingual speech. These projects 
have dealt with the reliability of Judges, their ability to 
specify speech characteristics, or their ability to Judge the ■ 
degree of accentedness and the proficiency of the speaker. 

In 1973, Gorosch compared oral EFL proficiency evaluations 
in Sweden by teachers, who were non-native speakers of English, 
and non-teachers, who were native speakers of English. His 
evaluators rated six Swedish EFL learners by noting each mistake 
in pronunciation and then they assessed overall intelligibility 
on a five-point scale. His data indicated that both groups 
tended to separate pronunciation from intelligibility and that 
the evaluations of jfehe non-teachers were unpredictable. He 
concluded that there are "considerable differences between 
assessments produced by teachers and those produced by non- 
teachers" (151). „ 

Contrary to Gorosch's findings, a study by Brennan, Ryan, 
and Dawson (1970) demonstrated that native speakers could give 
reliable Judgements. Seventy-two naive listeners Judged the 
degree of accentedness in 8 samples of Spanish-English. In 
addition to the high reliability, the results indicated that 
the Judges agreed on what degree of accentedness constituted 
a proficiency level. Further, it showed that the subjects during 
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an informal question period were unable to articulate which 
featul'es pf language were important to their judgements. 

Thrqugh^ 'a .1970 experiment, Giles attempted to arrange three 
supposed speech characteristics *hierarchially: the aesthetic, ^ 
tire communicative, and the status. These three characteristic? 
were defined as the pleasantness- of the voice, the intelligibility 
of the accent^, and the prestige value of the accent. His "subjects, 
adolescents from S^. Wales and S.W. England, listened to and rated 
13 Briti^ regional amd foreign accents on the three characteristics 
mentioned aboye. . His data showed 1;^'at althou^i the subjects 
were able to identify the individual accents, the three characteris- 
tics were apparently indistinguishable. He concluded that these 
characteristics were at best "three variants of one evaluative 
dimension" (219). 

Further research conducted by Galvan, Pierce, and Underwood 
.{1975) examined the speech of Mexican-American bilinguals in terms 
of 10 personality traits and 10 speech characteristics. Five 
recorded samples were evaluated by 92 Atnericaji undergraduates. 
The analysis showed that the raters generally evaluated speakers 
more positiv;ely than negatively, that the evaluations becsune 
more negative as accentedness increased, and that few differences 
could be predicted on the basis o'f the listeners' academic ^ 
backgrounds. In regards to the ^scales of speech characteristics 
(relaxed, appropriate, natural, standard, graceful, careful, 
understandable, good English, active, and smooth), they speculated 
that not only could the characteristics be reduced to a few factors 
but from these factors the points on an accent continuum could be 
established (i5) . 

The previously mentioned studies all had one thing in* common. 
They dealt with the speech of fairly established groups , whether 
these groups were bidialectal or bilingual. A study by Palmer 
(1973), however, dealt with a transient group of learners. Palmer 
pr,4uced a preliminary report of the subjective evaluations of 
ESL -^earners by naive listeners. Eighteen students from Georgetown 
Unives ty listened to 36 speedcers from four different languagi 
backgrounds (Lingatla, Arabic, Spanish, and Vietnamese). Each judge 
evaluated keach speaker on a five-point scale across three tasks: 
reading, r#^telling, and narration. The Judges reliably rated the 
speakers ac oss language backgrounds and across tasks. However, 
the raters were "not very good in identifying the source language*^ 
background of the speaker. It would seem from his; results that 
particular foi ign accents, i.e. Spanish, may no^ be as distinctive 
as popularly bv.^ieved. 

This study attempts to address a number of remaining questions. 
Here, as in Palmer's study, the focus is on learners of English 
as a second langu -^^e; but unlike Palmer's study, -this one compares 
interrater reliability among naive raters with interrater reliability 

among experienced ESL teachers. Further, evaluations by both groups 
are evaluated agair 't the independent placement of the speakers in 
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one of five proficiency levels, via, a separate testing procedure. 
Whereas the earlier studies were concerned with the effect of a 
speaker's accent on the listener's judgement without knowing what 
characteristics of speech contributed to that evaluation, this 
experiment vill attempt to separate accentedness into distinguishable 
dimensions. 

In particular, the following questions are addressed; 

(1) How much agreement is there on the evaluation of samples 
of non-native speech among Judges? 

(2) Is there any difference between naive native raters and 
experienced ESL teachers in the evaluations of foreign 
speech? 

(3) Will evaluations of non-native speech samples correlate 
with the independent placement of the speakers in an 
intensive English program, that is, do the ratings have 
concurrent validity? 

(k) Are there distinguishable dimensions for the ratings ff 

ac^entefiness in this study? 
(5) How accurately can the Judges identify the source 

language background of each speaker? 

METHOD 

Speech Samples . The experimenter asked instructors from 
each of the five proficiency levels at the Center for English 
as a Second Language (CESL), Southern Illinois University^ to 
recommend students of average speaking ability from Arabic, 
Persian, and Spanish speaking language backgrounds who would be j 
willing to participate in a short recording session. Two native / 
speakers who were graduate students in the Departmenx of Linguis- . 
tics. Southern Illinois University, were also taped. Originally 
25 ESL students, along with the two American students, were re- 
corded in a laboratory setting reading one of twelve 100-word I 
passages in English. {The paragraphs are given in Appendix A.) 
Each speaker was allowed to practice the passage twice before he 
was recorded. Eleven tape samples were eliminated due to excessive 
noise or to speaker duplication. Three non-natives from each of 
the five levels of CESL (15 ESL students) were finally selected - 
along with one of the' American students. 

Questionnaire: Scales of Accentedness and Overall Proficiency , 

The scales were c/)nstructed in a semantic differential form 
similar to the scales used by Lambert, Galvan, Palmer, and their 
colleagues. The first four scales consisted of four pairs of bi- 
polar adjectival descriptors, "not very intelligible" to "in- 
telligible," "unpleasant" to "pleasant," "unacceptable" to 
"acceptable," and* "non-native" to "native." In addition, there 
was an ovierall proficiency scale (OPS). Each of these scales 
was in a six-point Likert type format. Also, a multiple choice 
question about the language background of the speaker was in- 
cluded. (The questionnaire is given in Appendix B.) 

Raters . The tape and questionnaire were administered to 
70 raters (Rs). Half of them were enrolled in undergraduate 
English composition courses and had neither liuguistic nor 
-teaching experience (the naive group), while the other half were 



■ D 

166 



instructors or teaching assistants in ESL (the experienced group). 
Before taking the test, each R completed a form on biographical 
data. The naive Rs were tested during their regular class meetings 
while the teachers were tested either individually or in small 
Toups\ ^ 

Rating Procedure . Each sample was rated by the naive native 
judges-^and the experienced ESL teachers on each of the six scales. 
The ordeV of the six scales was the some for all samples of speech. 
The order oTxthe l6 speakers, however, was randomized for the first 
tape and was gl^en in reverse order on a second tape. Detailed in- 
structions for th^^use of the protocols were presented orally with 
an example using a male Spanish speaker. Form A was used with 

15 of the ESL teachers^^a^d 20 of the naive judges. Form B was 
used for 15 of the naive Jiidges and 20 of the ESL teachers . In 
sum, each of the Rs heard thfev^ame directions, either orally or 
taped, and the same example, amisthe;> listened to one of the two 
tapes of the l6 speakers , They evaluated each speaker on each 
of the six scales, 

RESULTS \ 

Rater Agreement , In order to see vhetl^r or not the raters 
agree in their evaluations of the speaker's abqentedness , -judges* 
ratings we';e treated as variables in a Q-type fkec^r analysis. • 
Normally, of course, the variables input to a f actoT^analysis 
are test scores, scales, or other measures. In this case, the 
raters were treated as variables and the various accent >Qales 
were treated as subfiles (each containing l6 cases, i,e, tn^^ 

16 speech -samples ) in th-e data set, (Due to computer space 
limitations, only 6o Rs could be included. Five were excluded \^ 
from each group on 'a random basis,) In the first factor analysis, 
the first four scales were treated together without distinction. 

We will^k^shortly return to the Justification for this (See thf 
discussion of question U, below,). In the principal components 
analysis. Factor 1 accounted for kQ% of the variance among the 
individual Rs , All of the Rs loaded positively on this factor 
and above ,36^Table 1,) Fifty-six Rs showed a correlation of 
greater than .50 with this factor, while 12 of these 56 loaded at 
.80 or higher. The overall mean loading was .69. The mean . 
loading for the naive Rs was .6^*5, and for the 30 SSL teachers 
was ,735* On a similar principal components analysis for the 
OPS, the first factor accounted for 56% of the total variance. 
The average loading for i;he naive Rs was .676; for the ESL 
teachers^ it was ,8l6 (Table 2), From these analyses, it can 
be concluded that there is a very substantial agreement among 
the Rs, regardless of whether they are naive or experienced. 

Difference in Reliability . The second question was whether 
a significant difference between naive Rs and experienced ESL 
teachers in the overall reliability of their ratings existed, . 
The answer to this question can be deduced directly from the 
^loadings of the previously defined factors. We can simply 
contrast the average loading of the naive Rs with the avera^^e 
loading of the ESL teachers. The contrast between che naive and th^ 
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Table 1 



The First Factor from a Q-Type Principal Components Analysis 
of the Four Scales Combined (Intelligibility, Pleasantness, 
Acceptability, and Natlveness) . ■ 

iSive Loadings on- Experienced Loadings on 

Rater Factor 1 Rater Factor 1 



7 .500 . 37 



811 



8 .U66 30 ^ -691 

9 .8kO 39 -505 
.81»9 ho ' • .810 
.805 hi „ .778 



10 
11 

.12 , 
13 
.lit 

^6 . 580 ho .700 



.718 h2 . .752 

.^29 U3 . , .833 

.661* . hh ^ . .666 

.767 ^+5 •T'*8 



17 .698 ' 1*7 • '^^^ 

18 .lt2U hQ .693 

19 , .721 h9 .778 



20 * .589 50 



21 .539 

22 



51 .596 

.791 52 .809 

23 - .373 53 -739 

2U '' .1*92 51* . .791 
25 



.69U . 55 -'f 2 

26 .'(77 ' 56 .^Oij 

27 
28 

29 
30 
31 

32 .703 
33 
3U 
-31 



.525 57 

Mh 58 ■ .812 

.655 . 59 . fl 

.719 bO • .71t 

.687 61 .785 

703 62 .681 

!*06 63 - . '.837 

,5li7 Gk .73li 

.616 iliiL 



"^The first and the last five .raters were eliminated so as not to 
exceed the space limitation of the SPSS factor program (P'Al, 
Nie, Hull, Jenkins, Steinbrenner , and Bent, 1975, li79-l»8o) . 
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^ Table ,2 

The First Factor from a Q-Type Principal Components Analysis 
of the Overall Proficiency Scale'. 



iicii, V e 






Loadings 


Rater 




rvct 


Factor 1 


\j 


OS? 


36 


.620 


7 


.673 


37 


.731 


8 


. S16 


38 


.729 


0 


* iy I 


39 


.83U 


10 


• 895 


liO 


.863 


11 


'.727 


lil 


.715 


12 


.81i8 


k2 


.717 


1'^ 


.778 


1*3 


.91U 


lU 




l|li 


* i i i 




.830 


li5 


.827 


l6 


.8li6 


li6 


.8OU 


17 


.660 


liY 


.783 


l8 


.361 


li8 


.82ii 


10 


.698 


kg 

^y 


.850 


20 


■.598 


" 50 


.881 


21 


.6U2 


5'' 


.633 




01 s 




.873 


23 


.39^ 


53 


.723 


?x 


Ml 


fl* 


.81i2 


25 


.917 


'55 


.792 


26 


.658 


56 


.808 


27 


.615 


57 


.796 


28 


.357 


58 


.81il 


29 


.781* 


59 


.762 


30 


.570 


60 


.786 


31 


.766 


61 


.801 


32 


.729 _^ ,, 


62 


.810 


33 


.326 


b3 


.760 


3»* 


.613 


61i 


.828 


35 


.651* 


65'* 


.708 



* See the footnote with Table 1. 
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experienced Rs is not significant at the ,05 level for the four 
scales lumped together, but the contrast for the OPS is^ signigi- 
cant at the .05 level. Therefore, the experienced Rs appear to 
be somewhat' more_ reliable, although it should be noted that both 
groups are surprisingly reliable on the whole. This can be in- 
ferred from the strenght of, the loadings on the two principal 
factors defined above. 

Concurrent Validity . Question 3 concerned the concurrent 
validity o^ the accent ratings. A convenient criterion was the 
speaker's placement at CESL. The overall correlation between 
the OPS ajld the various placement levels was .66. For the 
five Arabic speakers, separately, the correlation was .87* 
For the, five Spanish speakers, it was .71 and for fne five 
P^reiaiis, it was only Obviously, the overall ratings were 

VK>]:e in agreement for the Arab subgroup. 

Dimensions of Accentedness . Question k asks 'whether the 
——-- -'^ several scales are actually^v sources of unique variance with 
respect to ratings of accentedness or overall proficiency. 
Table 2 shows the intercorrelations between the four scales and 
the OPS with the 70 Rs times the 16 speakers as input data. 



Table 3. 

InterGorrelations ajnong the Scales of Accentedness 

' Scaxes 12 3 h ^ 5 

1. Intelligibility 1.000 TjQl 7m ^89 

2. Pleasantness 1.000 .786 .727 - -790 

3. Acceptability l-'OOO .792 .892 
U. Kativeness 1-000 .802 
5. Overall Proficiency 1.000 



Each scale appears to be measuring the same thing as the other 
scale. When mean scores (Table h) are input for the speakers 
on^each scale the correlation across the scales approximates 
unity — that is, they are almost perfect, never less than .97. 

Table? k. 

Mean Correlations of the Scales of Acoentedness 

Scales 1 2 3 h 3 

1. Intelligibility 1.000 -996 -998 -Tl -997 

2. Pleasantness i.OOO .99^* -970 .993 

3. Acceptability 1.000 .971* .978 
k. Nativeneas 1.000 .978 
5. Overall Proficiency 1.000 
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Thus, wc may conclude tha^ the scales, in this study, for practiced 
purposes ore unitaxy, . 

Idelitif ication of Langua>ge Backgrounds , ' The- fi-fth question 
asked' how accurately judges would be -able to p.dentify each iipeaker^s* 
language background. The experienced Bs correctly identified the 
source language backgrounds 73% of the tim^,, while the naive 
Rs coul(J do so only 30% of the time* Identification^ of .the 
individual language backgrounds by the experienced' R\, was best 
for the Spanish speaker,s , ' then the Persians, and then the Arabs, 
For the. naive Rs , the 'Order was'* Spanish, Arabs, and I^f^rsians 
(Tabic 5). • 

Table > . ' - ' 

Tne Percentages of Correctly Identified Language liaokgrounds 

Lunguagc Experienced Naive 

Arabic .66.3 ' 

L3panish '6h .0 32,6 

Persian 68.6 26,9 

Combined 73.0 30,0 

Of these three languages only Spanish was studied by more 
than one R (6 naive Rs and 20 experienced Rs). For the naive 
Rs it is interesting to note that those who had not haA Spanish 
identified the speakers with l6% greater accuracy than the naive 
Judges who had studied Spanish (Table 6), In looking at the, 
ratings of the experienced Rs, we discover that with or without 
studying Spanish, Rs could identify the speakers equally well. , 
It seems, therefore, that the ability to coi*rectly identify the 
language background of a speaicer may be due more to mere , contact 
with the speaker of the language in question than to formal 
study of the language. 

Table 6. 

The Percentages of Correctly Identified Spanish Speakers 

Experienced Naive 

Those who had 

studied LJpanisn ' > 8b 20 

Tnose who had not 

studiea c'panish 6ii 36 



DISCUSSION 

With little or no research basis, Cartier speculated that teach- 
ers are not the most reliable judges of oral proficiency. With 
equally little empirical study, Jakobovits suggested that naive 
natives are also not the best-^ judges . Obviously, both Cartier 
and Jakobovits cannot be right, but neither recommended reseal'ch 
to test their claims. With unabashed certainty, language testers 
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have categorized oral performance, inl^ the', separate ;components 

of accent, grammar, /vocabulary, fluency, and comprel\efision 

iVaiette, 1967; Harris, -1969$ Clark, ^ 1^72; Heaton, 1975;.. 

bavies, 1977'^. These categorlea'have become the sanctioned 

criteria for teachers' evaluations^of ' prof icie"ncy •/ ^^^ir 

empirical necessity has go?lfe;unqu6aiiQU?d,^ for*the ii^ost,, * 'i 

part * ' • 

*The data from this experiment disputes both: st)eculation 
of Cartier and. the opposite claim of Jakobovitsj .for if^they 
were true, we would expect to find little reliability in -the 
evaluations of either of the two groups. However, the results 
indicate that both groups can distinguish degrees of proficiency 
■with substantial reliability, although the teachers are somewhat 
more reliable than' the naive Judges- On^the other hand, the latter 
fact does not support Jakobovits* claim because of the demonstrated 
unity of the various scales of accentedness and the overall 
proficiency rating. Apparently, all of the raters tended to 
make wholistic unidimensional evaluations, rather than the mulvi- 
dimensional and separate evaluations of the presumed components. ; 
The unity of the scales suggests that dividing oral performance 
into components is superfluous at best, and artif actual at worst. 
According to the available empirical evidence, a listener does 
not and, indeed cannot component ialize the characteristics of ' 
speech, blather it would appear that the overall comprehensib1.lity 
is what moti- ates . the evaluation. 

The incjility of the naive Judge to' identify the source 
language backgrounds of the speakers substantiates Palmer' s 
findings (1973). The data, also, show that ESL 'teachers , the 
experienced group, are quite sucessful ,(X355) in identifying' 
source language backgrounds. Further ^ the data indicate^ con- 
trary to Palmer (1973), that accents are quite distinctive. 

More research is required before it will be possible to 
relate degrees of accentedness to points on a well-defined, 
although subjective, continuum. In addition, experiments should 
be conducted to see if the ability to identify a speaker^s first 
language affects the reliability and validity of the evaluations. 
Since studies have shown that speech characteristics may .affect 
personality assessments the converse relationship between personalitjf^ 
assessments and speech characteristics on the reliability and 
validity of evaluations should also be investigated. 

A final area of research would be to see how ndn-nativetj might 
evaluate oral performance in ESL. This study used native speakers 
of English as raters: what if we wanted to generalize to .non-natives 
from different language backgrounds? Would Arabs, for example, tend' 
to rate Arabic speakers in the same way as they might rate native 
speakers of Spanish or Persian? . Would they be more lenient? 
Would-they be as equally reliable? 

^ This' is an expanded jrers ion of a paper presented at The First 
International Conference on Frontiers in Langua,'e Proficiency and 
Dominance Testing at Southern Illinois University, Carbondale 
Illinois or April 22, 1977. ^ f 
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Appendix A 



The texts used in- this study were selected from several 
sources • Each passage vas rewri"Cten to meet a 100-word criterion. 
Texts 1, 2, 3, and U were each i sad by two different speakers , 

1. Ben got off the bus 'and then the bus drove away. He 
forgot about the tickecs because it was raining. The road was wet 
and there was a very big hole in hia shoe. Then a second bus 
stepped and he got on. This time there was a seat. He paid a 
dime for his ticket and then shut his eyes. ^ When he opened them 
again, the bus was past the- theater. He rang- t^h^ bell and the 
bus stopped suddenly. It was still raining as he walked back 

to the thdHlSei* and went in through the door. He saw many 

photographs of the actors just before he saw the stage. 

(Adapted from Baird, Brcughton, Cartwright, and Roberts, 1972, kk%) 

2. I hope to learn several foreign ieuiguages but English 
is the one I want to study first.- To begin with, I hope to get 
a good position with* one of the big companies in the capital and 

it will be an advantage for me to have an understanding of English. 
.If my work should ever require ray travelling outside of the country « 
it would be helpful if I knew English. It is used in carrying 
on business in almost every part of the world. My brothers and 
sisters, already skillful in English, are eager to practice it 
with me,", so I will have many opportunities when I am ready to 
speak English. 

(Adapted from Van Syoc and Van Syoc, 1971, 89.) 

3» On Saturday mornings the big pub^'-^.c library opens at 
half past nine. 'A lot of the people gO into the library on 
Saturday because this is the time when they go shopping, they 
take the ir^ books into -^he library, and ^o home with new ones. 
Susan and Mary, the two girl librarians, were standing behind 
the desk'. ' They took the books from the people who came in and 
gave them their tickets • It was va warm Saturday, and a lot of 
people were .in the streets and 'in the stores', and many wore 
coming into the libra'iy too. v 

(Adapted frcan Baird, Brough'ton, Cartwright, and Roberts,' 1972, U7.) 

' ,* k. As Wfcis expected the favorites had gotten well out in 
front with the remaining horses grouped together some way ^behind. 
On a dangei^ous^bend', three of the horses leading the group' fell, 
throwing the riders into gi'eat confusion. As the race progressed, 
the track became full Qf horaes without riders. Towards the end, 
there were only three horses left.* College Joy and Sweet Seven- 
teen were still leading the rabe with an unknown horse far behind. 
The crowd was very disappointed when on the last jump m the race, 
the riders -of both favorites failed to keep in the saddle. The 
crowd cheered and applauded as the unknown horse crossed the 
finishing line . 

(Adapted from 'Alexander, 197^, 66.) 

5. Moving the pilot aside, the man took his seat and listened 
carefully to the urgent** instructions that were being sent* by 
radio from the airport below. The plane was how dangerously close 

III 
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to the ground, but it soon began to climb. The man had to circle 
the airport several times in order to become familiar with the con- 
trols. The terrible moment came when he had to land the plane. 
Following the instructions, the man guided the plane towards the 
airfield. It shook violently as it touched the ground and then 
moved rapidly across the field, but after a long run it stopped 
safely. 

(Adapted from Alexander, 197^, 

6. The following Sunday we stayed at home, even thoug}i it 
was G fine day. About noon a large and very expensive car stopped 
outside our house. We were astonished when we saw several people 
preparing to have a picnic in our small garden, Father got 

very angry and went out to ask them what they thought they wt:re 
doing. You can imagine his surprise when he recognized the man 
who had taken our address the week before. Both men burst out^ 
laughing and father welcou^ed the strangers into the house. In 
time, we became friends, but we had learned a lesson we have 
never forgotten. 

(Adapted from Alexander, 197^, 63.) 

7. It was a very dark and stormy night. Two men were 
walking slowly down the road. Snow was covering the ground and 
a cold wind was blowing. They noticed a light behind some 
trees and soon arrived at a house. A poor old man immediately 
invited them into a clean room. He seemed & strange fellow, 
but he spoke kindly and offered them milk and fresh x>uit. 

The men remained there until morning. Then the man led them 
to the nearest town, but he would not accept ar^ money f or - 
his help. 

(Adapted from Alexander, 197^ > lO-) 

8. Science has told us so much about the moon that it 
is fairly easy to imagine what it would be like to go there • 

It is certainly not a friendly place. As there is no air or water, 
there can be no life of any kind. Also for mile after mile, 
there are only flat plains of dust with mountains around them. 
Above, the sun and stars shine in a black sky. The. moon is very 
silent. But beyond the horizon, our earth is shining more brdghtly 
than the stars. It looks like an immense ball, coloied blue, 
green, and brown. • 

(Adapted from Alexander, 3 97^, 35.) . , 

9. The store was empty and very peaceful. We sat down in 
the main hall and listened to the rain beating against the 
windows. Suddenly there was a loud noise at the door. Then 

a large party of boys were led in by. a teacher. The poor man 
was trying to keep them quiet, but they werfe not paying any 
attention to him. The boys ran here and there. The teacher ' 
explained that the-boys were rather excited* But the noise 
proved too much for us, so we decided to leave. After all, 
the boys had more right to the store than we did. 
(Adapted from Alexander, 197^, 5^.) 

10. Driving along a highway one dark night, Tom suddenly 
had a flat tire. Even worse, he discovered that he did not have 

a spare tire in the back of his car. Tom waved to passing cars and 
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trucks, but not one of them stopped. At last, he waved to a 
cer like his ovm. To his surprise, the car actually stopped and 
a well-dressed woman got out. The woman offered him her spare 
tire, but Tom had never changed a tire in his life. So she 
set to work at once and changed the tire in a few minutes while 
Tom looked on, 

(Adapted from Alexander, 197^, 27.) 

11, Dan found the school work easy. He read widely both 
at school and in the branch library. After the third year of 
high school, he left to take a job with a glass firm. Art work 
had always been a major interest, and he did so well with the 
firm that he was promised rapid advancement. But then the 
depression came, the business failed, and Dan was without a Job. 
At first, he went out looking for a Job and continued his art 
work at home, but when all his efforts brought no results,, he 
stopped looking for work and even lost interest in art, 
(Adapted fran Whyte, 1955, ^^-9-) 

12. Tony came into the club to talk the situation over 
with John. He was trying to get transpoxtlation, he said, but 
even if he could arrange it in the next few minutes it was so 
late that the boys would 'miss a large part of the evening. If 
anyone wanted his ^money back or a ticket for the next football 
game, he could have it. John explained the situation to the 
boys and then said that he thou^t it would be better if we went 
another time. Tony agreed. He said that John could collect the 

'tickets later. 
(Adapted from^ Whyte, 1955-, l82.) 



Appendix B 
QUESTIONNAIRE 



Name ^_ 

State or Country^ 

Native Lg. 

Other Lgs. 

Age 



Sex 



M 



ESL Expei*ience Yrs . 

Mos , 



In this experiment, you. will rate how well some non-native 



speakers read a short prose passage 
identify their native language. . 

EXAlviPLE 

NVI (Not Very Intelligible) 1 
UNP (Not Pleasant) 1 
Ui^A (Not Acceptable) 1 
NN (Non-native) 1 
OPS'^ f 1 

Language Background Ar. 
Ar. - Arabic, Sp. - Spanish, Pr. - 
X - Unknown 

i4 



In addition, you a^e asked to 
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^Overall Oral Proficiency Scale (OPS) 
!• The speaker is unintelligible to a native speaker. 
2* 'ilie speaker has a very heavy accent and makes frequent gross 
errors. 

3. The speaker's accent requires concentrated listening. His ^' 
mispronunciations lead to occasional misunderstandings. 
The foreign accent is evident and occasional mispronunciations 
occur, but these do not interfere with understanding. 

5. There are no consistent mispronunciations, but because of 
occasional deviations would not be taken for a native speaker. 

6. The speaker has native pronunciation with no trace of a 
foreign accent. 
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11. NVI 
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